AMENDMENTS TO THE CLAIMS 

Please amend the claims of the present application as set forth below in the 
following claim listing. A status identifier is provided for each claim in a 
parenthetical expression following each claim number. Changes to the claims are 
shown by strikethrough (for deleted matter) or underlining (for added matter). 

Claim Listing: 

1. (Cancelled). 

2. (Currently amended) One or more computer readable media as r e cit e d 
in claim 1, having stored thereon a plurality of instructions that, when executed by 
one or more processors, causes the one or more processors to perform acts 
including: 

receiving an initial image selection; 

generating a plurality of query vectors by extracting, for each query vector, 
one of a plurality of low-level features from the initial image selection; 

selecting a set of potentially relevant images based at least in part on 
distances between the plurality of query vectors and a plurality of feature vectors 
corresponding to low-level features of a plurality of images; 

receiving feedback regarding the relevance of one or more images of the set 
of potentially relevant images; 

generating a new plurality of query vectors based at least in part on the 
feedback; 

generating a weighting of feature elements based at least in part on the 
feedback; and 
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selecting a new set of potentially relevant images based at least in part on 
both the weighting of feature elements and distances between the new plurality of 
query vectors and the plurality of feature vectors, wherein the selecting a new set 
of potentially relevant images comprises using a matrix in determining the 
distance between one of the new plurality of query vectors and one of the plurality 
of feature vectors, and further comprising dynamically selecting the matrix based 
on both a number of images in the set of potentially relevant images for which 
relevance feedback was input and a number of feature elements in the one feature 
vector. 

3. (Original) One or more computer readable media as recited in claim 2, 
wherein the dynamically selecting comprises using a diagonal matrix if the 
number of images in the set of potentially relevant images for which relevance 
feedback was input is less than the number of feature elements in the one feature 
vector, and otherwise using a full matrix. 

4. (Original) One or more computer readable media as recited in claim 2, 
wherein the dynamically selecting comprises: 

if the number of images in the set of potentially relevant images for which 
relevance feedback was input is not less than the number of feature elements in the 
one feature vector, then using one matrix that transforms the query vector and the 
one feature vector to a higher-level feature space and then using another matrix 
that assigns a weight to each element of the transformed query vector and the 
transformed feature vector; and 
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if the number of images in the set of potentially relevant images is less than 
the number of feature elements in the one feature vector, then using a matrix that 
assigns a weight to each element of the query vector and the one feature vector. 

5. (Previously presented) One or more computer readable media as recited 
in claim 2, wherein X represents an image matrix that is generated by stacking N 
feature vectors, each of length K, corresponding to the set of potentially relevant 
images for which relevance feedback was received and resulting in an (AT x K) 
matrix, C represents a weighted covariance matrix of X , det(C) represents the 
matrix determinant of C, and the matrix comprises a full matrix (W*) that is 
generated as follows: 

2^ 

W* = (det(C)) K C- 1 . 

6. (Previously presented) One or more computer readable media as recited 
in claim 2, wherein represents the kk xh element of matrix W , x k represents the 
k ih feature element, oy represents the standard deviation of the sequence of x k % 
the matrix comprises a diagonal matrix with each diagonal element being 
generated as follows: 

1 

w k k = — • 
°k 

7. (Currently amended) One or more computer readable media as r e cit e d 
in claim 1 having stored thereon a plurality of instructions that, when executed by 
one or more processors, causes the one or more processors to perform acts 
including: 
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receiving an initial image selection; 

generating a plurality of query vectors by extracting, for each query vector, 
one of a plurality of low-level features from the initial image selection; 

selecting a set of potentially relevant images based at least in part on 
distances between the plurality of query vectors and a plurality of feature vectors 
corresponding to low-level features of a plurality of images; 

receiving feedback regarding the relevance of one or more images of the set 
of potentially relevant images; 

generating a new plurality of query vectors based at least in part on the 
feedback , wherein N represents the number of images in the set of potentially 
relevant images for which relevance feedback has been received, it n represents the 
relevance of image n in the set of images, n represents a transposition of a vector 
generated by concatenating the individual n n values, and X represents an image 

matrix that is generated by stacking N training vectors corresponding to the set of 

— > 

potentially relevant images into a matrix, and wherein each new query vector ( q ) 
of the new plurality of query vectors is generated as follows: 

- n X 
9=^ 1 

n=] 

generating a weighting of feature elements based at least in part on the 
feedback; and 

selecting a new set of potentially relevant images based at least in part on 
both the weighting of feature elements and distances between the new plurality of 
query vectors and the plurality of feature vectors . 
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8. (Previously presented) One or more computer readable media having 
stored thereon a plurality of instructions that, when executed by one or more 
processors, causes the one or more processors to perform acts including: 

receiving an initial image selection; 

generating a plurality of query vectors by extracting, for each query vector, 
one of a plurality of low-level features from the initial image selection; 

selecting a set of potentially relevant images based at least in part on 
distances between the plurality of query vectors and a plurality of feature vectors 
corresponding to low-level features of a plurality of images; 

receiving feedback regarding the relevance of one or more images of the set 
of potentially relevant images; 

generating a new plurality of query vectors based at least in part on the 
feedback; 

generating a weighting of feature elements based at least in part on the 
feedback; and 

selecting a new set of potentially relevant images based at least in part on 
both the weighting of feature elements and distances between the new plurality of 
query vectors and the plurality of feature vectors, wherein f. represents a 
summation, over the images in the set of potentially relevant images, of a product 
of a relevance of the image and a distance between the query vector and the 
feature vector, and wherein the selecting a new set of potentially relevant images 
comprises combining, for each image, a weighted distance between the plurality of 
query vectors and the plurality of feature vectors, and wherein the weight ( u i ) for 
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each of a plurality (/) of distances between a query vector and a corresponding 
feature vector is calculated as: 



9. (Cancelled). 

10. (Currently amended) One or more computer readable media as rocitod 
in claim 1 having stored thereon a plurality of instructions that, when executed by 
one or more processors, causes the one or more processors to perform acts 
including: 

receiving an initial image selection; 

generating a plurality of query vectors by extracting, for each query vector, 
one of a plurality of low-level features from the initial image selection , wherein 
the low-level features include: a color moments feature, a wavelet based texture 
feature, and a water-fill edge feature; 

selecting a set of potentially relevant images based at least in part on 
distances between the plurality of query vectors and a plurality of feature vectors 
corresponding to low-level features of a plurality of images; 

receiving feedback regarding the relevance of one or more images of the set 
of potentially relevant images; 

generating a new plurality of query vectors based at least in part on the 
feedback; 

generating a weighting of feature elements based at least in part on the 
feedback; and 




7 



Application No. 09/631,854 



selecting a new set of potentially relevant images based at least in part on 
both the weighting of feature elements and distances between the new plurality of 
query vectors and the plurality of feature vectors . 

1 1 . (Original) A method of selecting between two types of matrixes to be 
used to weight, based on relevance feedback, a plurality of feature elements for 
image retrieval, the method comprising: 

selecting one of the two types of matrixes based on both a number of 
previously retrieved relevant images and a length of a feature vector including the 
plurality of feature elements. 

12. (Original) A method as recited in claim 11, wherein the selecting 
comprises selecting one of the two types of matrixes based on both a number of 
previously retrieved potentially relevant images which were identified by a user as 
being relevant, and the length of the feature vector including the plurality of 
feature elements. 

13. (Original) A method as recited in claim 11, wherein the plurality of 
feature elements are all elements of the same feature. 

14. (Original) A method as recited in claim 11, wherein the selecting 
comprises using a first type of matrix if the number of retrieved relevant images is 
less than the length of the feature vector, and otherwise using a second type of 
matrix. 
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15. (Original) A method as recited in claim 14, wherein the first type of 
matrix comprises a diagonal matrix and wherein the second type of matrix 
comprises a full matrix. 

16. (Original) A method as recited in claim 11, wherein the selecting 
comprises using a first type of matrix if the length of the feature vector exceeds 
the number of retrieved relevant images by at least a threshold amount, and 
otherwise using a second type of matrix. 

17. (Original) A method as recited in claim 16, wherein the first type of 
matrix comprises a full matrix and the second type of matrix comprises a diagonal 
matrix. 

18. (Original) One or more computer readable media including a computer 
program that is executable by a processor to perform the method recited in claim 
11. 

19. (Previously presented) One or more computer readable media having 
stored thereon a plurality of instructions that, when executed by one or more 
processors, causes the one or more processors to perform acts including: 

comparing, for each of a plurality of images, a plurality of feature elements 
from a query vector to a plurality of feature elements from a feature vector 
corresponding to the image; 
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identifying a number of potentially relevant images based on the 
comparing; 

receiving user feedback regarding relevancy of one or more of the 
potentially relevant images; 

re-comparing, for each of the plurality of images, the plurality of feature 
elements from the query vector to the plurality of feature elements from the 
feature vector, including using a matrix to compare the feature elements and 
dynamically selecting a type of matrix to use based on both the user feedback and 
the number of the plurality of feature elements; 

identifying a new set of potentially relevant images based on the re- 
comparing; and 

presenting the new set of potentially relevant images to the user. 

20. (Original) One or more computer readable media as recited in claim 
19, wherein the re-comparing comprises dynamically selecting the type of matrix 
to use based on both a number of the potentially relevant images for which user 
feedback has been received and the number of the plurality of feature elements. 

21. (Original) One or more computer readable media as recited in claim 
19, wherein the dynamically weighting comprises using a first type of matrix if the 
number of retrieved relevant images is less than the length of the feature vector, 
and otherwise using a second type of matrix. 
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22. (Original) One or more computer readable media as recited in 21, 
wherein the first type of matrix comprises a diagonal matrix and the second type 
of matrix comprises a full matrix. 

23. (Original) A method comprising: 

generating a query vector corresponding to a feature of one image; 
identifying a feature vector corresponding to the feature of another image; 
identifying a number of training samples for which relevance feedback has 
been received; 

if the number of training samples either equals or exceeds a threshold 
amount, then determining a distance between the query vector and the feature 
vector including transforming the query vector and the feature vector to a higher- 
level feature space and then assigning a weight to each element of the transformed 
query vector and the transformed feature vector; and 

if the number of training samples does not exceed the threshold amount, 
then determining the distance between the query vector and the feature vector 
including assigning a weight to each element of the query vector and the feature 
vector. 

24. (Original) A method as recited in claim 23, wherein the feature vector 
includes a plurality of feature elements and wherein the threshold amount 
comprises the number of feature elements in the feature vector. 
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25. (Original) A method as recited in claim 23, wherein if the number of 

training samples either equals or exceeds the threshold amount, then determining 

— ► — > 
the distance ( g ), where P is a mapping matrix, q is the query vector, x is the 

feature vector, and A is a weighting matrix, as: 

g = (P(q-h T MP(q-h- 

26. (Original) A method as recited in claim 23, wherein if the number of 
training samples does not exceed the threshold amount, then determining the 
distance ( g ), where q is the query vector, jc is the feature vector, and A is a 
weighting matrix, as: 

£ = (?-*) r A(?-x). 

27. (Previously presented) A method as recited in claim 23, further 
comprising: 

repeating the generating, identifying of the feature vector, identifying of the 
number of training samples, and the determining for each of a plurality of features; 
and 

identifying how closely the image and the another image match each other 
by combining the distances between the query vectors and the feature vectors for 
the plurality of features. 

28. (Previously presented) A method as recited in claim 27, wherein the 
identifying how closely the image and the another image match each other 
comprises calculating a weighted summation of each of the individual distances 
for each of the plurality of features. 
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29. (Original) One or more computer readable media including a computer 
program that is executable by a processor to perform the method recited in claim 
23. 

30. (Original) A system comprising: 

a query vector generator to generate a query vector corresponding to a 

feature of one image; 

a comparator, coupled to the query vector generator, to, 

identify a feature vector corresponding to the feature of another image, 

identify a number of training samples for which relevance feedback has 

been received, 

if the number of training samples either equals or exceeds a threshold 
amount, then to determine a distance between the query vector and the feature 
vector including transforming the query vector and the feature vector to a higher- 
level feature space and then assigning a weight to each element of the transformed 
query vector and the transformed feature vector, and 

if the number of training samples does not exceed the threshold amount, 
then to determine the distance between the query vector and the feature vector 
including assigning a weight to each element of the query vector and the feature 
vector. 

31. (Cancelled). 
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32. (Previously presented) A method comprising: 

for one of a plurality of images and each of a plurality of features, 
generating, based on a set of search criteria, a query vector for the feature, 
identifying a feature vector, corresponding to the image, for the feature, and 
determining how closely the feature vector matches the query vector; and 
determining how closely the image matches the set of search criteria based 
on how closely, for the plurality of features, the feature vectors match the query 
vectors, wherein generating the query vector comprises generating the query 
vector based at least in part on user relevance feedback regarding how relevant 
images previously displayed to a user were. 

33. (Currently amended) A method as r e cit e d in claim 32, comprising: 
for one of a plurality of images and each of a plurality of features, 
generating, based on a set of search criteria, a query vector for the feature, 
identifying a feature vector, corresponding to the image, for the feature, 

wherein identifying the feature vector compris e s includes : 

identifying a low-level feature vector corresponding to the feature; 

and 

mapping the low-level feature vector to a higher level feature space; 
determining how closely the feature vector matches the query vector; and 
determining how closely the image matches the set of search criteria based 
on how closely, for the plurality of features, the feature vectors match the query 
vectors, wherein generating the query vector comprises generating the query 
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vector based at least in part on user relevance feedback regarding how relevant 
images previously displayed to a user were. 

34. (Original) A method as recited in claim 33, wherein the identifying the 
feature vector further comprises incorporating, into the mapping, relevance 
feedback. 

Claims 35-45 (Cancelled). 

46. (Original) A method of generating a query vector to compare to a 
feature vector of another image, the method comprising: 

receiving feedback regarding the relevance of each image of a set of 
images; 

wherein N represents the number of images in the set of images for which 
user relevance feedback has been received, n n represents the relevance of image n 
in the set of images, n represents a transposition of a vector generated by 
concatenating the individual n n values, and X represents an image matrix that is 
generated by stacking TV training vectors corresponding to the set of images into a 
matrix; and 

generating a query vector (q) corresponding to one of a plurality of 
features as follows: 

- K X 

<7=^ ■ 
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47. (Original) One or more computer readable media including a computer 
program that is executable by a processor to perform the method recited in claim 



48. (Original) A method of generating a weight to apply to distances 
between query vectors and feature vectors when combining the distances, the 
method comprising: 

receiving feedback regarding the relevance of each image of a set of 
images; 

wherein f. represents a summation, over the images in the set of images, of 
a product of a relevance of the image and a distance between the query vector and 
the feature vector; and 

generating a weight ( u i ) for each of a plurality ( / ) of distances between a 
query vector corresponding to one of a plurality ( / ) of features and a feature 
vector corresponding to the one of the plurality ( / ) of features as: 



49. (Original) One or more computer readable media including a computer 
program that is executable by a processor to perform the method recited in claim . 
48. 

50. (Cancelled). 

51. (Cancelled). 



46. 
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52. (Currently amended) One or more computer readable media as recit e d 
in claim 1 having stored thereon a plurality of instructions that, when executed by 
one or more processors, causes the one or more processors to perform acts 
including: 

receiving an initial image selection; 

generating a plurality of query vectors by extracting, for each query vector, 
one of a plurality of low-level features from the initial image selection; 

selecting a set of potentially relevant images based at least in part on 
distances between the plurality of query vectors and a plurality of feature vectors 
corresponding to low-level features of a plurality of images; 

receiving feedback regarding the relevance of one or more images of the set 
of potentially relevant images , wherein the receiving feedback compris e s includes 
receiving feedback via speech recognition ; and 

generating a new plurality of query vectors based at least in part on the 
feedback . 

53. (Original) One or more computer readable media as recited in claim 
19, wherein the receiving user feedback regarding relevancy comprises receiving 
user feedback in a range including at least Highly Relevant, Relevant, No Opinion, 
Irrelevant, and Highly Irrelevant. 

54. (Original) One or more computer readable media as recited in claim 
19, wherein the receiving user feedback comprises receiving user feedback via 
speech recognition. 
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55. (Previously presented) One or more computer readable media 
including a computer program that is executable by a processor to cause the 
processor to perform acts of: 

receiving user feedback regarding the relevance of each image of a set of 
images, the user feedback forming a range including at least Highly Relevant, 
Relevant, No Opinion, Irrelevant, and Highly Irrelevant; 

wherein N represents the number of images in the set of images for which user 

feedback has been received, n n represents the relevance of image n in the set of 

images, n represents a transposition of a vector generated by concatenating the 

individual n n values, and X represents an image matrix that is generated by 

stacking AT training vectors corresponding to the set of images into a matrix; and 

— > 

generating a query vector ( q ) corresponding to one of a plurality of features as 
follows: 

"> K X 

4=^ • 

56. (Original) One or more computer readable media as recited in claim 
55, wherein the receiving user feedback comprises receiving user feedback via 
speech recognition. 

57. (Previously presented) One or more computer readable media having 
stored thereon a plurality of instructions that, when executed by one or more 
processors, causes the one or more processors to: 
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select a set of potentially relevant images based at least in part on distances 
between a plurality of query vectors extracted from an initial image selection and a 
plurality of feature vectors corresponding to low-level features of a plurality of 
images; 

receive feedback regarding the relevance of one or more images of the set 
of potentially relevant images; 

generate a new plurality of query vectors based at least in part on the 
feedback; 

generate a weighting of feature elements based at least in part on the 
feedback; and 

select a new set of potentially relevant images based at least in part on both 
the weighting of feature elements and distances between the new plurality of query 
vectors and the plurality of feature vectors, wherein f. represents a summation, 
over the images in the set of potentially relevant images, of a product of a 
relevance of the image and a distance between the query vector and the feature 
vector, and wherein the selecting a new set of potentially relevant images 
comprises combining, for each image, a weighted distance between the plurality of 
query vectors and the plurality of feature vectors, and wherein the weight ( u i ) for 
each of a plurality ( / ) of distances between a query vector and a corresponding 
feature vector is calculated as: 
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58. (Previously presented) One or more computer readable media as 
recited in claim 57, wherein the plurality of instructions to cause the one or more 
processors to select comprises instructions to cause the one or more processors to 
use a matrix in determining the distance between one of the new plurality of query 
vectors and one of the plurality of feature vectors, and further comprises 
instructions to cause the one or more processors to dynamically select the matrix 
based on both a number of images in the set of potentially relevant images for 
which relevance feedback was input and a number of feature elements in the one 
feature vector, wherein the instructions to dynamically select comprise instructions 
to cause the one or more processors to use a diagonal matrix when the number of 
images in the set of potentially relevant images for which relevance feedback was 
input is less than a number of feature elements in the one feature vector, and 
otherwise using a full matrix. 

59. (Previously presented) One or more computer readable media as 
recited in claim 57, wherein the plurality of instructions to cause the one or more 
processors to select comprises instructions to cause the one or more processors to: 

use one matrix that transforms the query vector and the one feature vector 
to a higher-level feature space and then using another matrix that assigns a weight 
to each element of the transformed query vector and the transformed feature vector 
when the number of images in the set of potentially relevant images for which 
relevance feedback was input is not less than the number of feature elements in the 
one feature vector; and 



20 



Application No. 09/631,854 



use a matrix that assigns a weight to each element of the query vector and 
the one feature vector when the number of images in the set of potentially relevant 
images is less than the number of feature elements in the one feature vector. 

60. (Previously presented) One or more computer readable media 
including computer readable instructions executable by one or more processors to 
cause the one or more processors to select between two types of matrixes to be 
used to weight, based on relevance feedback, a plurality of feature elements for 
image retrieval, wherein the instructions are further executable to cause the one or 
more processors to select one of the two types of matrixes based on both a number 
of previously retrieved relevant images and a length of a feature vector including 
the plurality of feature elements. 

61. (Previously presented) One or more computer readable media as 
recited in claim 60, wherein instructions executable to cause the one or more 
processors to select comprise instructions executable to cause the one or more 
processors to select one of the two types of matrixes based on both a number of 
previously retrieved potentially relevant images which were identified by a user as 
being relevant, and the length of the feature vector including the plurality of 
feature elements. 

62. (Previously presented) One or more computer readable media as 
recited in claim 60, wherein the plurality of feature elements are all elements of 
the same feature. 
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63. (Previously presented) One or more computer readable media as 
recited in claim 60, wherein instructions executable to cause the one or more 
processors to select comprise instructions executable to cause the one or more 
processors to use a first type of matrix when the number of retrieved relevant 
images is less than the length of the feature vector, and otherwise use a full matrix. 

64. (Previously presented) One or more computer readable media as 
recited in claim 60, wherein instructions executable to cause the one or more 
processors to select comprise instructions executable to cause the one or more 
processors to use a diagonal matrix when the number of retrieved relevant images 
is less than the length of the feature vector, and otherwise use another type of 
matrix. 

65. (Previously presented) One or more computer readable media as 
recited in claim 60, wherein instructions executable to cause the one or more 
processors to select comprise instructions executable to cause the one or more 
processors to use a full matrix when the length of the feature vector exceeds the 
number of retrieved relevant images by at least a threshold amount, and otherwise 
use another type of matrix. 

66. (Previously presented) One or more computer readable media as 
recited in claim 60, wherein instructions executable to cause the one or more 
processors to select comprise instructions executable to cause the one or more 
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processors to use a full matrix when the length of the feature vector exceeds the 
number of retrieved relevant images by at least a threshold amount, and otherwise 
use a diagonal matrix. 
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